Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add cross device sync #1005

Merged
merged 32 commits into from
Mar 16, 2024
Merged

Conversation

kaiserbh
Copy link

This essentially refers to the pull request I submitted at GitHub - Tachiyomi Pull Request #9790. My initial thought was to contribute the pull request to the upstream project, anticipating that it would subsequently be integrated into the downstream, particularly SY, which I predominantly use. However, given the halt in development for the vanilla version, I opted to directly submit the pull request to SY. This PR encompasses numerous modifications; the feature in question has been available for a considerable duration and has undergone thorough testing on the Discord server, with all functionalities operating smoothly.

Currently, there are two services available: Syncyomi, which is self-hosted, and Google Drive.

Once merged you will need to create OAuth 2.0 Client IDs in google console then replace the app/src/main/assets/client_secrets.json

image

Once all of that is done just need to set the app to publish instead of testing that way everyone can access it otherwise limited to 100 users we add for testing, we don't need to submit the app for approval since we are not using sensitive scopes. More info here

OAuth Consent Screen

image

Scopes

image

image
image
image
image
image

I forgot to add the data into the merging logic, So remote always have empty value :(. Better late than never.
Various improvement and added the option to choose what they want to sync. Added sync library button to LibraryTab as well.

Signed-off-by: KaiserBh <kaiserbh@proton.me>
Signed-off-by: KaiserBh <kaiserbh@proton.me>
@kaiserbh
Copy link
Author

Opened a PR to upstream mihonapp/mihon#40 so feel free to wait for that to be merged before merging this.

@jobobby04
Copy link
Owner

Great! Hopefully it's merged soon

@NeKoOuO
Copy link

NeKoOuO commented Jan 16, 2024

@kaiserbh omg I just saw an issue from seven years ago talking about the need for cloud sync. I didn't expect it to actually exist now. Thank you for your greatful PR

Also changed it to be app specific, we don't want them to use sync data from SY or other forks as some of the model and backup is different. So if people using other forks they should use the same data and not mismatch.
I think we were reaching deadlock or infinite loop causing the sync to go forever.
Using timestamp to sync is a bit skewed due to system clock etc and therefore there was a lot of issues with it such as removing a manga that shouldn't have been removed. Marking chapters as unread even though it was marked as a read. Hopefully by using versioning system it should eliminate those issues.
Signed-off-by: KaiserBh <kaiserbh@proton.me>
@kaiserbh
Copy link
Author

Hi @jobobby04, I have closed the PR to the upstream I explained it on the discord announcement for the people who are testing it or been using it for the past year ish, anyway, I know the PR is big and I am more than happy to streamline it and divide it to make it smaller. So whenever you got the time can you do initial review of it and let me know the changes I need to make?

Copy link
Owner

@jobobby04 jobobby04 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good, some nitpicks here and there

Signed-off-by: KaiserBh <kaiserbh@proton.me>
We don't need this anymore since we are utilizing versioning system.

Signed-off-by: KaiserBh <kaiserbh@proton.me>
I forgot to cherry pick this from mihon branch.
Signed-off-by: KaiserBh <kaiserbh@proton.me>
Signed-off-by: KaiserBh <kaiserbh@proton.me>
Signed-off-by: KaiserBh <kaiserbh@proton.me>
Signed-off-by: KaiserBh <kaiserbh@proton.me>
When there is changes in the chapters table such as reading or updating last read page we should bump the manga version. This way the manga is synced properly as in the History and last_read history is synced properly. This should fix the sorting issue.

Signed-off-by: KaiserBh <kaiserbh@proton.me>
Signed-off-by: KaiserBh <kaiserbh@proton.me>
@kaiserbh kaiserbh requested a review from jobobby04 March 15, 2024 12:28
@jobobby04
Copy link
Owner

I've pushed the latest Mihon commits, looks like conflicts need fixing

Signed-off-by: KaiserBh <kaiserbh@proton.me>
@kaiserbh
Copy link
Author

I've pushed the latest Mihon commits, looks like conflicts need fixing

fixed.

@jobobby04 jobobby04 merged commit 334e9fb into jobobby04:master Mar 16, 2024
@NeKoOuO NeKoOuO mentioned this pull request Mar 17, 2024
cuong-tran referenced this pull request in komikku-app/komikku Mar 18, 2024
* feat: add cross device sync.

* chore: add google api.

* chore: add SY specifics.

* feat: add backupSource, backupPref, and "SY" backupSavedSearches.

I forgot to add the data into the merging logic, So remote always have empty value :(. Better late than never.

* feat(sync): Allow to choose what to sync.

Various improvement and added the option to choose what they want to sync. Added sync library button to LibraryTab as well.

Signed-off-by: KaiserBh <kaiserbh@proton.me>

* oops.

Signed-off-by: KaiserBh <kaiserbh@proton.me>

* refactor: fix up the sync triggers, and update imports.

* refactor

* chore: review pointers.

* refactor: update imports

* refactor: add more error guard for gdrive.

Also changed it to be app specific, we don't want them to use sync data from SY or other forks as some of the model and backup is different. So if people using other forks they should use the same data and not mismatch.

* fix: conflict and refactor.

* refactor: update imports.

* chore: fix some of detekt error.

* refactor: add breaks and max retries.

I think we were reaching deadlock or infinite loop causing the sync to go forever.

* feat: db changes to accommodate new syncing logic.

Using timestamp to sync is a bit skewed due to system clock etc and therefore there was a lot of issues with it such as removing a manga that shouldn't have been removed. Marking chapters as unread even though it was marked as a read. Hopefully by using versioning system it should eliminate those issues.

* chore: add migrations

* chore: version and is_syncing fields.

* chore: add SY only stuff.

* fix: oops wrong index.

Signed-off-by: KaiserBh <kaiserbh@proton.me>

* chore: review pointers.

Signed-off-by: KaiserBh <kaiserbh@proton.me>

* chore: remove the option to reset timestamp

We don't need this anymore since we are utilizing versioning system.

Signed-off-by: KaiserBh <kaiserbh@proton.me>

* refactor: Forgot to use the new versioning system.

I forgot to cherry pick this from mihon branch.

* chore: remove isSyncing from Chapter/Manga model.

Signed-off-by: KaiserBh <kaiserbh@proton.me>

* chore: remove unused import.

Signed-off-by: KaiserBh <kaiserbh@proton.me>

* chore: remove isSyncing leftover.

Signed-off-by: KaiserBh <kaiserbh@proton.me>

* chore: remove isSyncing.

Signed-off-by: KaiserBh <kaiserbh@proton.me>

* refactor: make sure the manga version is bumped.

When there is changes in the chapters table such as reading or updating last read page we should bump the manga version. This way the manga is synced properly as in the History and last_read history is synced properly. This should fix the sorting issue.

Signed-off-by: KaiserBh <kaiserbh@proton.me>

---------

Signed-off-by: KaiserBh <kaiserbh@proton.me>
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 19, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants